Search for: All records

The data precision can significantly affect the accuracy and overhead metrics of hardware accelerators for different applications such as artificial neural networks (ANNs). This paper evaluates the inference and training of multi-layer perceptrons (MLPs), in which initially IEEE standard floating-point (FP) precisions (half, single and double) are utilized separately and then compared with mixed-precision FP formats. The mixed-precision calculations are investigated for three critical propagation modules (activation functions, weight updates, and accumulation units). Compared with applying a simple low-precision format, the mixed-precision format prevents an accuracy loss and the occurrence of overflow/underflow in the MLPs while potentially incurring in less hardware overhead in terms of area/power. As the multiply-accumulation is the most dominant operation in trending ANNs, a fully pipelined hardware implementation for the fused multiply-add units is proposed for different IEEE FP formats to achieve a very high operating frequency. 

This paper deals with the fault tolerance of Triplet Networks (TNs). Results based on extensive analysis and simulation by fault injection are presented for new schemes. As in accordance with technical literature, stuck-at faults are considered in the fault model for the training process. Simulation by fault injection shows that the TNs are not sensitive to this type of fault in the general case; however, an unexcepted failure (leading to network convergence to false solutions) can occur when the faults are in the negative subnetwork. Analysis for this specific case is provided and remedial solutions are proposed (namely the use of the loss function with regularized anchor outputs for stuck-at 0 faults and a modified margin for stuck-at 1/-1 faults). Simulation proves that false solutions can be very efficiently avoided by utilizing the proposed techniques. Random bit-flip faults are then considered in the fault model for the inference process. This paper analyzes the error caused by bit-flips on different bit positions in a TN with Floating-Point (FP) format and compares it with a fault- tolerant Stochastic Computing (SC) implementation. Analysis and simulation of the TNs confirm that the main degradation is caused by bit-flips on the exponent bits. Therefore, protection schemes are proposed to handle those errors; they replace least significant bits of the FP numbers with parity bits for both single- and multi-bit errors. The proposed methods achieve superior performance compared to other low-cost fault tolerant schemes found in the technical literature by reducing the classification accuracy loss of TNs by 96.76% (97.74%) for single-bit (multi-bit errors).  

Information is an integral part of the correct and reliable operation of today's computing systems. Data either stored or provided as input to computation processing modules must be tolerant to many externally and internally induced destructive phenomena such as soft errors and faults, often of a transient nature but also in large numbers, thus causing catastrophic system failures. Together with error tolerance, reliable operation must be provided by reducing the large overheads often encountered at system-level when employing redundancy. While information-based techniques can also be used in some of these schemes, the complexity and limited capabilities for implementing high order correction functions for decoding limit their application due to poor performance; therefore, N Modular Redundancy (NMR) is often employed. In NMR the correct output is given by majority voting among the N input copies of data. Reduced Precision Redundancy (RPR) has been advocated to reduce the redundancy, mostly for the case of N = 3; in a 3RPR scheme, one full precision (FP) input is needed while two inputs require reduced precision (RP) (usually by truncating some of the least significant bits (LSBs) in the input data). However, its decision logic is more complex than a 3MR scheme. This paper proposes a novel NRPR scheme with a simple comparison-based approach; the realistic case of N = 5 is considered as an example to explain in detail such proposed scheme; different arrangements for the redundancy (with three or four FP data copies) are considered. In addition to the design of the decision circuit, a probabilistic analysis is also pursued to determine the conditions by which RPR data is provided as output; it is shown that its probability is very small. Different applications of the proposed NRPR system are presented; in these applications, data is used either as memory output and/or for computing the discrete cosine transform. In both cases, the proposed 5RPR scheme shows considerable advantages in terms of redundancy management and reliable image processing.